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Abstract 

This papers presents a detailed analysis of pseudocodewords of Tanner graphs. Pseudocodewords arising on 
the iterative decoder's computation tree are distinguished from pseudocodewords arising on finite degree lifts. 
Lower bounds on the minimum pseudocodeword weight are presented for the BEC, BSC, and AWGN channel. 
Some structural properties of pseudocodewords are examined, and pseudocodewords and graph properties that 
are potentially problematic with min-sum iterative decoding are identified. An upper bound on the minimum 
degree lift needed to realize a particular irreducible lift-realizable pseudocodeword is given in terms of its 
maximal component, and it is shown that all irreducible lift-realizable pseudocodewords have components 
upper bounded by a finite value t that is dependent on the graph structure. Examples and different Tanner graph 
representations of individual codes are examined and the resulting pseudocodeword distributions and iterative 
decoding performances are analyzed. The results obtained provide some insights in relating the structure of 
the Tanner graph to the pseudocodeword distribution and suggest ways of designing Tanner graphs with good 
minimum pseudocodeword weight. 
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I. Introduction 

Iterative decoders have gained widespread attention due to their remarkable performance in decoding LDPC 
codes. However, analyzing their performance on finite length LDPC constraint graphs has nevertheless re- 
mained a formidable task. Wiberg's dissertation [1] was among the earliest works in characterizing iterative 
decoder convergence on finite-length LDPC constraint graphs or Tanner graphs. Both [1] and [2] examine the 
convergence behavior of the min-sum iterative decoder [3] on cycle codes, a special class of LDPC codes 
having only degree two variable nodes, and they provide some necessary and sufficient conditions for the 
decoder to converge. Analogous works in [4] and [5] explain the behavior of iterative decoders using the lifts 
of the base Tanner graph. The common underlying idea in all these works is the role of pseudocodewords in 
determining decoder convergence. 

Pseudocodewords of a Tanner graph play an analogous role in determining convergence of an iterative decoder 
as codewords do for a maximum likelihood decoder. The error performance of a decoder can be computed 
analytically using the distance distribution of the codewords in the code. Similarly, an iterative decoder's 
performance may be characterized by the pseudocodeword distance. Distance reduces to weight with respect 
to the all-zero codeword. Thus, in the context of iterative decoding, a minimum weight pseudocodeword [4] is 
more fundamental than a minimum weight codeword. In this paper we present lower bounds on the minimum 
pseudocodeword weight w m - m for the BSC and AWGN channel, and further, we bound the minimum weight 
of good and bad pseudocodewords separately. 

Paper [5] characterizes the set of pseudocodewords in terms of a polytope that includes pseudocodewords that 
are realizable on finite degree graph covers of the base Tanner graph, but does not include all pseudocodewords 
that can arise on the decoder's computation tree [1], [6]. In this paper, we investigate the usefulness of the 
graph-covers-polytope definition of [5], with respect to the min-sum iterative decoder, in characterizing the set 
of pseudocodewords of a Tanner graph. In particular, we give examples of computation trees that have several 
pseudocodeword configurations that may be bad for iterative decoding whereas the corresponding polytopes of 
these graph do not contain these bad pseudocodewords. We note however that this does not mean the polytope 
definition of pseudocodewords is inaccurate; rather, it is exact for the case of linear programming decoding 
[7], but incomplete for min-sum iterative decoding. 

As any pseudocodeword is a convex linear combination of a finite number of irreducible pseudocodewords, 
characterizing irreducible pseudocodewords is sufficient to describe the set of all pseudocodewords that can 
arise. It can be shown that the weight of any pseudocodeword is lower bounded by the minimum weight of 
its constituent irreducible pseudocodewords [5], implying that the irreducible pseudocodewords are the ones 
that are more likely to cause the decoder to fail to converge. We therefore examine the smallest lift degree 
needed to realize irreducible lift-realizable pseudocodewords. A bound on the minimum lift degree needed to 
realize a given pseudocodeword is given in terms of its maximal component. We show that all lift-realizable 
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irreducible pseudocodewords cannot have any component larger than some finite number t which depends on 
the structure of the graph. Examples of graphs with known t-values are presented. 

The results presented in the paper are highlighted through several examples. These include an LDPC 
constraint graph having all pseudocodewords with weight at least d m - m (the minimum distance of the code), 
an LDPC constraint graph with both good and low-weight (strictly less than d m - m ) bad pseudocodewords, 
and an LDPC constraint graph with all bad non-codeword-pseudocodewords. Furthermore, different graph 
representations of individual codes such as the [7,4,3] and [15, 11,3] Hamming codes are examined in order 
to understand what structural properties in the Tanner graph are important for the design of good LDPC codes. 
We observe that despite a very small girth, redundancy in the Tanner graph representations of these examples 
can improve the distribution of pseudocodewords in the graph and hence, iterative decoding performance. 

This paper is organized as follows. Definitions and terminology are introduced in Section 2. Lower bounds 
on the pseudocodeword weight of lift-realizable pseudocodewords are derived in Section 3, and a bound on the 
minimum lift degree needed to realize a particular lift-realizable irreducible pseudocodeword is given. Section 
4 presents examples of codes to illustrate the different types of pseudocodewords that can arise depending on 
the graph structure. Section 5 analyzes the structure of pseudocodewords realizable in lifts of general Tanner 
graphs. Finally, the importance of the graph representation chosen to represent a code is highlighted in Section 
6, where the [7,4,3] and [15,11,3] Hamming codes are used as case studies. Section 7 summarizes the results 
and concludes the paper. For readability, the proofs are given in the appendix. 

II. Background 

In this section we establish the necessary terminology and notation that will be used in this paper, including 
an overview of pseudocodeword interpretations, iterative decoding algorithms, and pseudocodeword weights. 
Let G = (V, U; E) be a bipartite graph comprising of vertex sets V and U, of sizes n and m, respectively, 
and edges E C {(v,u)\ v E V, u G U}. Let G represent a binary LDPC code C with minimum distance d m \ n . 
Then G is called a Tanner graph (or, LDPC constraint graph) of C. The vertices in V are called variable nodes 
and represent the codebits of the LDPC code and the vertices in U are called constraint nodes and represent 
the constraints imposed on the codebits of the LDPC code. 

Definition 2.1: A codeword c in an LDPC code C represented by a Tanner graph G is a binary assignment 
to the variable nodes of G such that every constraint node is connected to an even number of variable nodes 
having value 1, i.e., all the parity check constraints are satisfied. 

A. Pseudocodewords 

1 ) Computation Tree Interpretation: Wiberg originally formulated pseudocodewords in terms of the compu- 
tation tree, as described in [1], and this work was extended by Frey et al in [6]. Let C(G) be the computation 
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Fig. 1. A graph and its computation tree after two iterations of message passing. 



tree, corresponding to the min-sum iterative decoder, of the base LDPC constraint graph G [1]. The tree is 
formed by enumerating the Tanner graph from an arbitrary variable node, called the root of the tree, down 
through the desired number of layers corresponding to decoding iterations. A computation tree enumerated for 
£ iterations and having variable node V{ acting as the root node of the tree is denoted by Ci(G)f. The shape 
of the computation tree is dependent on the scheduling of message passing used by the iterative decoder on 
the Tanner graph G. In Figure ffj the computation tree C2(G)2 is shown for the flooding schedule [8]. (The 
variable nodes are the shaded circles and the constraint nodes are the square boxes.) Since iterative decoding 
is exact on cycle-free graphs, the computation tree is a valuable tool in the exact analysis of iterative decoding 
on finite-length LDPC codes with cycles. 

A binary assignment to all the variable nodes in the computation tree is said to be valid if every constraint 
node in the tree is connected to an even number of variable nodes having value 1. A codeword c in the 
original Tanner graph G corresponds to a valid assignment on the computation tree, where for each i, all 
nodes representing V{ in the computation tree are assigned the same value. A pseudocodeword p, on the 
other hand, is a valid assignment on the computation tree, where for each i, the nodes representing Vi in the 
computation tree need not be assigned the same value. 

For a computation tree C(G), we define a local configuration at a check Uj in the original Tanner graph G 
as the average of the local codeword configurations at all copies of Uj on C(G). A valid binary assignment 
on C(G) is said to be consistent if all the local configurations are consistent. That is, if a variable node V{ 
participates in constraint nodes u a and u^, then the coordinates that correspond to Vi in the local configurations 
at u a and Uf,, respectively, are the same. It can be shown that a consistent valid binary assignment on the 
computation tree is a pseudocodeword that also lies in the polytope of [5] (see equation 0, and therefore 
realizable on a lift-graph of G. A consistent valid binary assignment on C(G) also has a compact length n 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



5 



G 

(base graph) 



G 

(a degree 4 cover) 





c = (1,1,1,1, 1,1,1,1,0,0,0,0) 



c = (1,1,0,0, 1,0,0,0,0,1,0,0) 



P = (1,1,0) 



p = (1/2, 1/4, 1/4) 



Fig. 2. A pseudocodeword in the base graph (or a valid codeword in a lift). 

vector representation p = (pi,P2 5 • • • ,Pn) such that if a check node u a has variable nodes v ai ,v a2 , . . . ,v ar as 
its neighbors, then projecting p onto the coordinates 01,02, ... ,a r yields the local configuration of u a on the 
computation tree C(G). An inconsistent but valid binary assignment on C(G) on the other hand has no such 
compact vector representation. The computation tree therefore contains pseudocodewords some of which that 
lie in the poly tope of [5] and some of which that do not. 

2) Graph Covers Definition: A degree £ cover (or, lift) G of G is defined in the following manner: 

Definition 2.2: A finite degree £ cover of G = (V, U; E) is a bipartite graph G where for each vertex 
Xi € V U U, there is a cloud Xi = {xi t ,Xi 2 , . . . ,xi e } of vertices in G, with deg(xi j ) = deg(xi) for all 
1 < j < 4 and for every (a:*, xj) £ E, there are £ edges from X, L to Xj in G connected in a 1 — 1 manner. 

Figure |2] shows a base graph G and a degree four cover of G. 

A codeword c in a lift graph G of a Tanner graph G is defined analogously as in Definition 12.11 

Definition 2.3: Suppose that c = (ci t i, 6^2, • • • , ci^, 62,1, . . . , 62,1, . . . ) is a codeword in the Tanner graph G 
representing a degree £ lift of G. A pseudocodeword p of G is a vector (pi,P2> • • • ,Pn) obtained by reducing 
a codeword c, of the code in the lift graph G, in the following way: 

c = (ci,i, . . . ,61^,62,1, • • • ,c 2 ,e, ...)-> (ci,i + ci i2 H r-ci^,c 2 ,i + c 2i2 H \-c 2 ,e, ■■■) = (pi,p 2 ,--- ,p n )=P, 

where p { = (c^i + q j2 H h Q^). 

Note that each component of the pseudocodeword is merely the number of 1 -valued variable nodes in the 
corresponding variable cloud of G, and that any codeword c is trivially a pseudocodeword as c is a valid 
codeword configuration in a degree-one lift. Pseudocodewords as in this definition are called lift-realizable 
pseudocodewords and also as unsealed pseudocodewords in [9]. We will use this definition of pseudocodewords 
throughout the paper unless mentioned otherwise. 

Remark 2.1: It can be shown that that the components of a lift-realizable pseudocodeword satisfy the 
following set of inequalities. At every constraint node Ui, that is connected to variable nodes Vi x ,Vi 2 , ... ,Vi r , 
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the pseudocodeword components satisfy 

Ph <Pi 2 +Pi 3 ^ h Pi T 

Pi 2 <Pn+Ph-\ h Pi r 

Pi r <Pi,+Pi 2 ^ \-Pi r -! (1) 

Definition 2.4: A pseudocodeword that does not correspond to a codeword in the base Tanner graph is called 
a non-codeword pseudocodeword, or nc-pseudocodeword, for short. 

3) Polytope Representation: The set of all pseudocodewords associated with a given Tanner graph G has 
an elegant geometric description [5], [7]. In [5], Koetter and Vontobel characterize the set of pseudocodewords 
via the fundamental cone. For each parity check j of degree 6j, let Cj denote the (Sj, Sj — 1, 2) simple parity 
check code, and let Pg. be a 2 <5j ~ 1 x 8j matrix with the rows being the codewords of Cj. The fundamental 
polytope at check j of a Tanner graph G is then defined as: 

P GC (Cj) = {uj G R 5 - 1 : lo = xP Sj ,x G R 2 ^\fJ < x { < l^a* = 1}, (2) 

i 

and the fundamental polytope of G is defined as: 

P GC (G) ={u;e R n : u m G P GC {C 3 ),j = 1, . . . ,m}, (3) 



We use the superscript GC to refer to pseudocodewords arising from graph covers and the notation wjv(j) 
to denote the vector lo restricted to the coordinates of the neighbors of check Cj. The fundamental polytope 
gives a compact characterization of all possible lift-realizable pseudocodewords of a given Tanner graph G. 
Removing multiplicities of vectors, the fundamental cone F(G) associated with G is obtained as: 

F(G) = {huj £l":w£ P GC (G),n > 0}. 

A lift-realizable pseudocodeword p as in Definition 12.31 corresponds to a point in the graph-covers polytope 

P GC {G). 

In [7], Feldman also uses a polytope to characterize the pseudocodewords in linear programming (LP) 
decoding and this polytope has striking similarities with the polytope of [5]. Let E(Cj) denote the set of all 
configurations that satisfy the code Cj (as defined above). Then the feasible set of the LP decoder is given by: 

P LP (G) = {c G R n : x 3 G R 2 ^ -1 , Yl, x o\S = 1,Q = Yl x ^ s V * £ 

s&E{c 3 ) seE(Cj), ies 

< x jt3 < 1, V5 G E(Cj),j G {1, . . . , m}} 
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Fig. 3. A stopping set S = {vq, v\, V3, V5} in G. 

Remark 2.2: It can be shown that the polytopes of [5] and [7] are identical, i.e., P GC (G) = P LP (G) [10]. 
Definition 2.5: The support of a vector x = (xi, . . . ,x n ), denoted supp(x), is the set of indices % where 

Definition 2.6: [11] A stopping set in G is a subset S of V where for each s S S, every neighbor of s is 
connected to S at least twice. 

The size of a stopping set S is equal to the number of elements in S. A stopping set is said to be minimal 
if there is no smaller sized nonempty stopping set contained within it. The smallest minimal stopping set is 
called a minimum stopping set, and its size is denoted by s m i n . Note that a minimum stopping set is not 
necessarily unique. Figure [3] shows a stopping set in the graph. Observe that {t>4, tv, t>s} and {^3,^5,^9} are 
two minimum stopping sets of size s m [ n = 3, whereas {vq, v\, v%, v$} is a minimal stopping set of size 4. 

On the erasure channel, pseudocodewords of G are essentially stopping sets in G [4], [5], [7] and thus, the 
non-convergence of the iterative decoder is attributed to the presence of stopping sets. Moreover, any stopping 
set can potentially prevent the iterative decoder from converging. 

One useful observation is that that the support of a lift-realizable pseudocodeword as in Definition 12.31 forms 
a stopping set in G. This is also implied in [5] and [7]. 

Lemma 2.1: The support of a lift-realizable pseudocodeword p of G is the incidence vector of a stopping 
set in G. 

Definition 2.7: A pseudocodeword p = (p\, . . . ,p n ) is irreducible if it cannot be written as a sum of two 
or more codewords or pseudocodewords. 

Note that irreducible pseudocodewords are called minimal pseudocodewords in [5] as they correspond to 
vertices of the polytope P GC (G), and in the scaled definition of pseudocodewords in [5], any pseudocodeword 
is a convex linear combination of these irreducible pseudocodewords. We will see in subsequent sections that 
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the irreducible pseudocodewords, as defined above, are the ones that can potentially cause the min-sum decoder 
to fail to converge. 

B. Pseudocodewords and Iterative Decoding Behavior 

The feature that makes LDPC codes attractive is the existence of computationally simple decoding algorithms. 
These algorithms either converge iteratively to a sub-optimal solution that may or may not be the maximum 
likelihood solution, or do not converge at all. The most common of these algorithms are the min-sum (MS) and 
the sum-product (SP) algorithms [3], [12]. These two algorithms are graph-based message-passing algorithms 
applied on the LDPC constraint graph. More recently, linear programming (LP) decoding has been applied to 
decode LDPC codes. Although LP decoding is more complex, it has the advantage that when it decodes to a 
codeword, the codeword is guaranteed to be the maximum-likelihood codeword (see [7]). 

A message -passing decoder exchanges messages along the edges of the code's constraint graph. For binary 
LDPC codes, the variable nodes assume the values one or zero; hence, a message can be represented either 
as the probability vector [po>Pi]> where po is the probability that the variable node assumes a value of 0, and 
pi is the probability that the variable node assumes a value of 1, or as a log-likelihood ratio (LLR) log(^), 
in which case the domain of the message is the entire real line R. 

Let c = (c\,...,c n ) be a codeword and let w = (wi, . . . , w n ) be the input to the decoder from the 
channel. That is, the log-likelihood ratios (LLR's) from the channel for the codebits v±, . . . , v n are wi, . . . , w n , 
respectively. Then the optimal maximum likelihood (ML) decoder estimates the codeword 

c* = argmm(ciWi + C2W2 + • • • + c n w n ) = arg mincw T . 

Let V be the set of all pseudocodewords (including all codewords) of the graph G. Then the graph-based 
min-sum (MS) decoder essentially estimates [5] 

* ■ T 

x = ar g mm xw . 

We will refer to the dot product xw T as the cost-function of the vector x with respect to the channel input vector 
w. Thus, the ML decoder estimates the codeword with the lowest cost whereas the sub-optimal graph-based 
iterative MS decoder estimates the pseudocodeword with the lowest cost. 

The SP decoder, like the MS decoder, is also a message passing decoder that operates on the constraint graph 
of the LDPC code. It is more accurate than the MS decoder as it takes into account all pseudocodewords of 
the given graph in its estimate. However, it is still sub-optimal compared to the ML decoder. Thus, its estimate 
may not not always correspond to a single codeword (as the ML decoder), or a single pseudocodeword (as the 
MS decoder). A complete description, along with the update rules, of the MS and SP decoders may be found 
in [3]. 
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In this paper we will focus our attention on the graph-based min-sum (MS) iterative decoder, since it is 
easier to analyze than the sum-product (SP) decoder. The following definition characterizes the iterative decoder 
behavior, providing conditions when the MS decoder may fail to converge to a valid codeword. 

Definition 2.8: [2] A pseudocodeword p-{pi,P2, ■ ■ ■ ,Pn) is good if for all input weight vectors w = 
(wi,W2, ■ ■ ■ ,w n ) to the min-sum iterative decoder, there is a codeword c that has lower overall cost than 
p, i.e., cw T < pw T . 

Definition 2.9: A pseudocodeword p is bad if there is a weight vector w such that for all codewords c, 



Note that a pseudocodeword that is bad on one channel is not necessarily bad on other channels since the set 
of weight vectors w that are possible depends on the channel. 

Suppose the all-zeros codeword is the maximum-likelihood (ML) codeword for an input weight vector w, 
then all non-zero codewords c have a positive cost, i.e., cw T > 0. In the case where the all-zeros codeword is 
the ML codeword, it is equivalent to say that a pseudocodeword p is bad if there is a weight vector w such 
that for all codewords c, cw T > but pw T < 0. 

As in classical coding where the distance between codewords affects error correction capabilities, the distance 
between pseudocodewords affects iterative decoding capabilities. Analogous to the classical case, the distance 
between a pseudocodeword and the all-zeros codeword is captured by weight. The weight of a pseudocodeword 
depends on the channel, as noted in the following definition. 

Definition 2.10: [4] Let p = (j?i,P2, ■ ■ ■ ,Pn) be a pseudocodeword of the code C represented by the Tanner 
graph G, and let e be the smallest number such that the sum of the e largest p^s is at least ^= lP * . Then the 
weight of p is: 

• wbec(p) = I supp(p)| for the binary erasure channel (BEC); 

• wbsc(p) f or the binary symmetric channel (BSC) is: 



Note that the weight of a pseudocodeword of G reduces to the traditional Hamming weight when the 
pseudocodeword is a codeword of G, and that the weight is invariant under scaling of a pseudocodeword. The 
minimum pseudocodeword weight of G is the minimum weight over all pseudocodewords of G and is denoted 
by w mm c for the BEC (and likewise, for other channels). 



T T 

cw > pw . 




where Yl e Pi * s tne sum °f tne e Ingest Pi's. 




for the additive white Gaussian noise (AWGN) channel. 
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Remark 2.3: The definition of pseudocodeword and pseudocodeword weights are the same for general- 
ized Tanner graphs, wherein the constraint nodes represent subcodes instead of simple parity-check nodes. 
The difference is that as the constraints impose more conditions to be satisfied, there are fewer possible 
nc-pseudocodewords. Therefore, a code represented by an LDPC constraint graph having stronger subcode 
constraints will have a larger minimum pseudocodeword weight than a code represented by the same LDPC 
constraint graph having weaker subcode constraints. 

C. Graph-Cover s-Polytope Approximation 

In this section, we examine the graph-covers-polytope definition of [5] in characterizing the set of pseu- 
docodewords of a Tanner graph with respect to min-sum iterative decoding. Consider the [4, 1, 4] -repetition 
code which has a Tanner graph representation as shown in Figure |4] The corresponding computation tree for 
three iterations of message passing is also shown in the figure. The only lift-realizable pseudocodewords for this 
graph are (0, 0, 0, 0) and (k, k, k, k), for some positive integer k; thus, this graph has no nc-pseudocodewords. 
Even on the computation tree, the only valid assignment assigns the same value for all the nodes on the 
computation tree. Therefore, there are no nc-pseudocodewords on the graph's computation tree as well. 




Fig. 4. A Tanner graph and computation tree (CT) for the [4,1,4] repetition code. 

Suppose we add a redundant check node to the graph, then we obtain a new LDPC constraint graph, shown 
in Figure [5j for the same code. Even on this graph, the only lift realizable pseudocodewords are (0, 0, 0, 0) 
and (k, k, k, k), for some positive integer k. Therefore the polytope of [5] contains (0, 0, 0, 0) and (1, 1, 1, 1) 
as the vertex points and has no bad pseudocodewords (as in Definition 12. 91 ). However, on the computation tree, 
there are several valid assignments that do not have an equivalent representation in the graph-covers-polytope. 
The assignment where all nodes on the computation tree are assigned the same value, say 1, (as highlighted 
in Figure [5j corresponds to a codeword in the code. For this assignment on the computation tree, the local 
configuration at check u\ is (1,1) corresponding to (vi,V2), at check U2 it is (1, 1) corresponding to (v2,Vs), 
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at check us it is (1, 1) corresponding to (t>3, v±), at check 114 it is (1, 1) corresponding to (vi, V4), and at check 
it5 it is (1, 1, 1, 1) corresponding to (v\, V2, U3, 774). Thus, the pseudocodeword vector (1, 1, 1, 1) corresponding 
to (v±,V2,V3,Vi) is consistent locally with all the local configurations at the individual check nodes. 

However, an assignment where some nodes are assigned different values compared to the rest (as highlighted 
in Figure [6]) corresponds to a nc pseudocodeword on the Tanner graph. For the assignment shown in Figure |6j 
the local configuration at check u\ is (1,1), corresponding to (v\,V2), as there are two check nodes u\ in 
the computation tree with (1,1) as the local codeword at each of them. Similarly, the local configuration at 
check U2 is (2/3,2/3), corresponding to (v2,Vs), as there are three U2 nodes on the computation tree, two of 
which have (1, 1) as the local codeword and the third which has (0,0) as the local codeword. Similarly, the 
local configuration at check U3 is (1/3, 1/3) corresponding to (^3,^4), the local configuration at check M4 is 
(1/2, 1/2) corresponding to (1^1,^4), and the local configuration at check 115 is (1/3, 1,0,2/3) corresponding 
to (vi,V2,vs,V4 : ). Thus, there is no pseudocodeword vector that is consistent locally with all the above local 
configurations at the individual check nodes. 

Clearly, as the computation tree grows with the number of decoding iterations, the number of nc-pseudocodewords 
in the graph grows exponentially with the depth of the tree. Thus, even in the simple case of the repetition 
code, the graph-covers-polytope of [5] fails to capture all min-sum-iterative-decoding-pseudocodewords of a 
Tanner graph. 





viv 3 v 4 v 3 v 3Vl v 2 v 4 Vl v 3 v 3 v 2 v 3V 



Fig. 5. Modified Tanner graph and CT for the [4,1,4] repetition code. 





00** • * 00 00 MOO 

v 1 v 3 v 4 v 3 VjVj v 2 v 4 Vl v 3 v 3 v 2 v 3Vl 



Fig. 6. Modified Tanner graph and CT for the [4,1,4] repetition code. 



Figure [7] shows the performance of MS iterative decoding on the constraint graphs of Figures |4] and [5] 
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when simulated over the binary input additive white Gaussian noise channel (BIAWGNC) with signal to noise 
ratio Ef,/N . The ML performance of the code is also shown as reference. With a maximum of 10 4 decoding 
iterations, the performance obtained by the iterative decoder on the single cycle constraint graph of Figure 0] is 
the same as the optimal ML performance (the two curves are one on top of the other), thereby confirming that 
the graph has no nc-pseudocodewords. The iterative decoding performance deteriorates when a new degree 
four check node is introduced as in Figure [5] (A significant fraction of detected errors, i.e., errors due to the 
decoder not being able to converge to any valid codeword within 10 4 iterations, were obtained upon simulation 
of this new graph.) 



[4,1,4] Repetition Code 



10" 



10' 



10" 



10 



original representation (single cycle) 
new representation (new deg 4 check) 
ML decoding 



(10000 decoding iterations) 




10 



5 6 7 

Eb/No (dB) 



10 



Fig. 7. Performance of different representations of [4,1,4] repetition code over the BIAWGNC. 



This example illustrates that the polytope P (G) does not capture the entire set of MS pseudocodewords 
on the computation tree. In general, we state the following results: 

Claim 2.1: A bipartite graph G representing an LDPC code C contains no irreducible nc-pseudocodewords 
on the computation tree C(G) of any depth if and only if either (i) G is a tree, or (ii) G contains only degree 
two check nodes. 

Claim 2.2: A bipartite graph G representing an LDPC code C contains either exactly one or zero irreducible 
lift-realizable nc-pseudocodewords if either (i) G is a tree, or (ii) there is at least one path between any two 
variable nodes in G that traverses only via check nodes having degree two. 
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Note that condition (ii) in Claim 12.21 states that if there is at least one path between every pair of variable 
nodes that has only degree two check nodes, then G contains at most one irreducible lift-realizable nc- 
pseudocodeword. However, condition (ii) in Claim 12.11 requires that every path between every pair of variable 
nodes has only degree two check nodes. 

For the rest of the paper, unless otherwise mentioned, we will restrict our analysis of pseudocodewords to 
the set of lift-realizable pseudocodewords as they have an elegant mathematical description in terms of the 
polytope P (G) that makes the analysis tractable. 



III. Bounds on minimal pseudocodeword weights 

In this section, we derive lower bounds on the pseudocodeword weight for the BSC and AWGN channel, 
following Definition 12.101 The support size of a pseudocodeword p has been shown to upper bound its weight 
on the BSC/ AWGN channel [4]. Hence, from Lemma |2~T1 it follows that w^^ AWGN < s m in- We establish 
the following lower bounds for the minimum pseudocodeword weight: 

Theorem 3.1: Let G be a regular bipartite graph with girth g and smallest left degree d. Then the minimal 
pseudocodeword weight is lower bounded by 

w bsc/awgn > ) l + d + d(d-l) + --- + d(d-l)^, f odd 



mm 



l + d-\ Vd(d- 1) V + {d- 1) V , | even 



Note that this lower bound holds analogously for the minimum distance <i m i n of G [13], and also for the size 
of the smallest stopping set, s m - m , in a graph with girth g and smallest left degree d [14]. 

For generalized LDPC codes, wherein the right nodes in G of degree k represent constraints of a [k,k' , ek] 
sub-codeQ, the above result is extended as: 

Theorem 3.2: Let G be a fc-right-regular bipartite graph with girth g and smallest left degree d and let the 
right nodes represent constraints of a [k, k', ek] subcode, and let x = (ek — 1). Then: 



BSC/ AWGN > . 



1 + dx + d(d - l)x 2 H Vd(d-l)~x~, f odd 

1 + dx H + d{d - 1) * i i + (d — 1) 4 x±, | even 



Definition 3.1: A stopping set for a generalized LDPC code using [k,k',ek] sub-code constraints may be 
defined as a set of variable nodes S whose neighbors are each connected at least ek times to S in G. 

'Note that ek and e are the minimum distance and the relative minimum distance, respectively of the sub-code. 
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This definition makes sense since an optimal decoder on an erasure channel can recover at most ek — 1 
erasures in a linear code of length k and minimum distance ek. Thus if all constraint nodes are connected to 
a set S, of variable nodes, at least ek times, and if all the bits in S are erased, then the iterative decoder will 
not be able to recover any erasure bit in S. Note that Definition 13.11 assumes there are no idle components 
in the subcode, i.e. components that are zero in all the codewords of the subcode. For the above definition 
of a stopping set in a generalized Tanner graph, the lower bound holds for s m i n also. That is, the minimum 
stopping set size s mm in a k -right-regular bipartite graph G with girth g and smallest left degree d, wherein 
the right nodes represent constraints of a [k, k', ek] subcode with no idle components, is lower bounded as: 




1 + dx + d(d - l)x 2 H h d(d - 1)^ , f odd 

1 + dx H h d(d — 1) 4 x 4 + (a — 1) 4 x-i , | even 



where x = (ek — 1). 

The max-fractional weight of a vector x = [xi, . . . ,x n ] is defined as w m ax-frac( x ) = ma* 1 a;' ■ The max- 
fractional weight of pseudocodewords in LP decoding (see [7]) characterizes the performance of the LP 
decoder, similar to the role of the pseudocodeword weight in MS decoding. It is worth noting that for any 
pseudocodeword p, the pseudocodeword weight of p on the BSC and AWGN channel relates to the max- 
fractional weight of p as follows: 

Lemma 3.1: For any pseudocodeword p, w BSC ^ AWGN (p) > w> m ax-frac(p)- 

It follows that w^^ AWGN > d^rac' trie max-fractional distance which is the minimum max-fractional 
weight over all p. Consequently, the bounds established in [7] for df^. are also lower bounds for w m - m . One 
such bound is given by the following theorem. 

Theorem 3.3: (Feldman [7]) Let deg^ (respectively, deg~) denote the smallest left degree (respectively, 
right degree) in a bipartite graph G. Let G be a factor graph with deg^~ > 3, deg~ > 2, and girth g, with 
g > 4. Then 

q&>(degr-l)W-\ 



Corollary 3.4: Let G be a factor graph with deg x > 3, deg r > 2, and girth g, with g > 4. Then 

BSC / AWGN ^ / j - 1 \r £ l-l 
w min > {de 9l -1)1 ^ I . 



Note that Corollary 13.41 which is essentially the result obtained in Theorem 13.11 makes sense due to the 
equivalence between the LP poly tope and GC poly tope (see Section 2). 
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Recall that any pseudocodeword can be expressed as a sum of irreducible pseudocodewords, and further, it 
has been shown in [5] that the weight of any pseudocodeword is lower bounded by the smallest weight of its 
constituent pseudocodewords. Therefore, given a graph G, it is useful to find the smallest lift degree needed to 
realize all irreducible lift-realizable pseudocodewords (and hence, also all minimum weight pseudocodewords). 

One parameter of interest is the maximum component t which can occur in any irreducible lift-realizable 
pseudocodeword of a given graph G, i.e., if a pseudocodeword p has a component larger than t, then p is 
reducible. 

Definition 3.2: Let G be a Tanner graph. Then the maximum component value an irreducible pseudocode- 
word of G can have is called the t-value of G, and will be denoted by t. 

We first show that for any finite bipartite graph, the following holds: 

Theorem 3.5: Every finite bipartite graph G representing a finite length LDPC code has a finite t. 

Theorem 3.6: Let G be an LDPC constraint graph with largest right degree <i+ and t-value t . That is, any 
irreducible lift-realizable pseudocodeword p = (pi, . . . ,p n ) of G has < pi < t, for i = 1, . . . , n. Then the 
smallest lift degree m m - m needed to realize all irreducible pseudocodewords of G satisfies 

2~2vi€N( Ul )Pi ^ te?+ 
m-min < max < — -, 

Uj Z Z 

where the maximum is over all check nodes uj in the graph and N(v,j) denotes the variable node neighbors 

Of Uj. 

If such a t is known, then Theorem 13.61 may be used to obtain the smallest lift degree needed to realize all 
irreducible lift-realizable pseudocodewords. This has great practical implications, for an upper bound on the 
lift degree needed to obtain a pseudocodeword of minimum weight would significantly lower the complexity 
of determining the minimum pseudocodeword weight w m i n . 

Corollary 3. 7: If p is any lift-realizable pseudocodeword and b is the maximum component, then the smallest 
lift degree needed to realize p is at most ^f-. 

Example 3.1: Some graphs with known i-values are: t < 2 for cycle codes [1], [2], t = 1 for LDPC 
codes whose Tanner graphs are trees, and t < 2 for LDPC graphs having a single cycle, and t < s for tail- 
biting trellis codes represented on Tanner- Wiberg-Loeliger graphs [1] with state-space sizes si,...,s m and 
s = max{si,...,s m }. 

We now bound the weight of a pseudocodeword p based on its maximal component value t and its support 
size | supp(p)|. 
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Lemma 3.2: Suppose in an LDPC constraint graph G every irreducible lift-realizable pseudocodeword p = 
(pi,P2, ■ ■ ■ ,Pn) with support set V has components < pi < t, for 1 < i < n, then: (a) w AWGN (p) > 
(1+t2) gL l)+2t in and (b) w BSC (p) > 

For many graphs, the i-value may be small and this makes the above lower bound large. Since the support of 
any pseudocodeword is a stopping set (Lemma |2. lb . w m m can be lower bounded in terms of s m j n and t. Thus, 
stopping sets are also important in the BSC and the AWGN channel. 

Further, we can bound the weight of good and bad pseudocodewords (see Definitions 12.81 12.9b separately, 
as shown below: 

Theorem 3.8: For an [n, k, d m i n ] code represented by an LDPC constraint graph G: (a) if p is a good 
pseudocodeword of G, then W B SC/AWGN ^ > w max _f rac (p) > d m i n , and (b) if p is a bad pseudocodeword 
[2] of G, then w BSC ' AWGN (p) > u; max -f r ac(p) > where t is as in the previous lemma. 



IV. Examples 

In this section we present three different examples of Tanner graphs which give rise to different types of 
pseudocodewords and examine their performance on the binary input additive white Gaussian noise channel 
(BIAWGNC), with signal to noise ratio E b /N Q , with MS, SP, and ML decoding. The MS and SP iterative 
decoding is performed for 50 decoding iterations on the respective LDPC constraint graphs. 

Example 4.1: Figure [8] shows a graph that has no pseudocodeword with weight less than d m i n on the BSC 
and AWGN channel. For this code (or more precisely, LDPC constraint graph), the minimum distance, the 
minimum stopping set size, and the minimum pseudocodeword weight on the AWGN channel, are all equal to 
4, i.e., d m - m = s m i n = w m i n = 4, and the t-value (see Definition 13.2b is 2. An irreducible nc-pseudocodeword 
with a component of value 2 may be observed by assigning value 1 to the nodes in the outer and inner rings 
and assigning value 2 to exactly one node in the middle ring, and zeros elsewhere. 

Figure [9] shows the performance of this code on a BIAWGNC with MS, SP, and ML decoding. It is evident 
that all three algorithms perform almost identically. Thus, this LDPC code does not have low weight (relative 
to the minimum distance) bad pseudocodewords, implying that the performance of the MS decoder, under i.i.d. 
Gaussian noise, will be close to the optimal ML performance. 



Example 4.2: Figure [10] shows a graph that has both good and bad pseudocodewords. Consider p = 
(1,0,1,1,1,1,3,0,0,1,1,1,1,0). Letting w = (1,0,0,0,0,0,-1,0,0,0,0,0,0,0), we obtain pw T = -2 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



17 




Fig. 8. A graph with all pseudocodewords having weight at least d n 



10 



10 





Min-Sum decoding 




-0- Sum-Prod decoding 
Max-Likld. decoding 















































4 5 6 7 8 9 10 11 

E JN (dB) 



Fig. 9. Performance of Example ET]- LDPC code: MS, SP, ML decoding over the BIAWGNC. 



and cw T > for all codewords c. Therefore, p is a bad pseudocodeword for min-sum iterative decoding. In 
particular, this pseudocodeword has a weight of w BSC ^ AWGN (p) = 8 on both the BSC and the AWGN channel. 
This LDPC graph results in an LDPC code of minimum distance d m j n = 8, whereas the minimum stopping 
set size and minimum pseudocodeword weight (AWGN channel) of the graph are 3, i.e., s m i n = w m - m = 3, 
and the t-value is 8. 

Figure rill shows the performance of this code on a BIAWGNC with MS, SP, and ML decoding. It is evident 
in the figure that the MS and the SP decoders are inferior in performance in comparison to the optimal ML 
decoder. Since the minimal pseudocodeword weight w m - m is significantly smaller than the minimum distance of 
the code d m i n , the performance of the MS iterative decoder at high signal to noise ratios (SNRs) is dominated 
by low-weight bad pseudocodewords. 
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Fig. 10. A graph with good and bad pseudocodewords. 
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Fig. 11. Performance of Example [42]- LDPC code: MS, SP, ML decoding over the BIAWGNC. 

Example 4.3: Figure [12] shows a graph on m + 1 variable nodes, where the set of all variable nodes 
except v\ form a minimal stopping set of size 777, i.e., s mm — 777. When 777 is even, the only irreducible 
pseudocodewords are of the form (k, 1,1,..., 1), where < k < m and k is even, and the only nonzero 
codeword is (0, 1, 1, ... , 1). When m is odd, the irreducible pseudocodewords have the form (k, 1, 1, ... , 1), 
where 1 < k < m, and k is odd, or (0, 2,2,..., 2), and the only nonzero codeword is (1,1,..., 1). In general, 
any pseudocodeword of this graph is a linear combination of these irreducible pseudocodewords. When k is 
not or 1, then these are irreducible nc -pseudocodewords; the weight vector w = (wi, . . . ,w m+ i), where 
w\ = —1, W2 = +1, and W3 = ■ ■ ■ = w m+ i = 0, shows that these pseudocodewords are bad. When 777 is even 
or odd, any reducible pseudocodeword of this graph that includes at least one irreducible nc -pseudocodeword in 
its sum, is also bad (according to Definition I2.9I ). We also observe that for both the BSC and AWGN channel, 
all of the irreducible pseudocodewords have weight at most d m [ n = 777 or 777 + 1, depending on whether m 
is even or odd. The minimum pseudocodeword weight is w^^ GN = 4777/(777 + 1), and the LDPC constraint 
graph has a t-value of m. 

Figures [13] and Q3] show the performance of the code for odd and even 777, respectively, on a BIAWGNC 



- Min-Sum decoding 
Sum-Prod decoding 
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Fig. 12. A graph with only bad nc-pseudocodewords. 

with MS, SP, and ML decoding. The performance difference between the MS (respectively, the SP) decoder 
and the optimal ML decoder is more pronounced for odd m. (In the case of even m, (0, 2, 2 ... , 2) is not a 
bad pseudocodeword, since it is twice a codeword, unlike in the case for odd m; thus, one can argue that, 
relatively, there are a fewer number of bad pseudocodewords when m is even.) Since the graph has low weight 
bad pseudocodewords, in comparison to the minimum distance, the performance of the MS decoder in the 
high SNR regime is clearly inferior to that of the ML decoder. 



Figure [15] shows the performance of Example 3 for m = 10 and m = 11 over the BSC channel with 
MS iterative decoding. Since there are only 2 m+1 different error patterns possible for the BSC channel, the 
performance of MS decoding for each error pattern was determined and the average number of output errors 
were computed. The figure shows that all four-bit or less error patterns were corrected by the MS and the ML 
decoders. However, the average number of output bit errors for five-bit error patterns with MS decoding was 
around 0.48 for the m = 11 code and was around 5.55 for the m = 10 code, while the ML decoder corrected 
all five-bit error patterns for both the codes. The average number of output bit errors for six-bit error patterns 
with MS decoding was 6 for the m = 11 code and 10.5 for the m = 10 code, whereas the ML decoder 
corrected all six bit errors for the m = 11 code and yielded an average number of output bit errors of 4.545 
for the m = 10 code. The figure also shows that MS decoding is closer to ML decoding for the m = 10 code 
than for the m = 11 code. 

This section has demonstrated three particular LDPC constraint graphs having different types of pseu- 
docodewords, leading to different performances with iterative decoding in comparison to optimal decoding. In 
particular, we observe that the presence of low weight irreducible nc-pseudocodewords, with weight relatively 
smaller than the minimum distance of the code, can adversely affect the performance of iterative decoding. 

V. Structure of pseudocodewords 

This section examines the structure of lift-realizable pseudocodewords and identifies some sufficient condi- 
tions for certain pseudocodewords to potentially cause the min-sum iterative decoder to fail to converge to a 
codeword. Some of these conditions relate to subgraphs of the base Tanner graph. We recall that we are only 
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Fig. 13. Performance of Example 1431- LDPC code for m = 11: MS, SP, ML decoding over the BIAWGNC. 
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Fig. 14. Performance of Example 03]- LDPC code for m = 10: MS, SP, ML decoding over the BIAWGNC. 



considering the set of lift-realizable pseudocodewords and that by Definition 12.31 the pseudocodewords have 
non-negative integer components, and hence are unsealed. 

Lemma 5.1: Let p = (pi,P2, ■ ■ ■ ,Pn) be a pseudocodeword in the graph G that represents the LDPC code 
C. Then the vector x = p mod 2, obtained by reducing the entries in p, modulo 2, corresponds to a codeword 
in C. 



The following implications follow from the above lemma: 

• If a pseudocodeword p has at least one odd component, then it has at least <i m i n odd components. 

• If a pseudocodeword p has a support size | supp(p)| < d m i n , then it has no odd components. 

• If a pseudocodeword p does not contain the support of any non-zero codeword in its support, then p has 
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Fig. 15. Performance of Example I4.31 LDPC code for m = 11, m = 12 with MS and ML decoding over the BSC. 



no odd components. 

Lemma 5.2: A pseudocodeword p = (p%, . . . ,p n ) can be written as p = + c^ 2 ) + • • • + + r, where 
cW , . . . , c^ fc ) , are k (not necessarily distinct) codewords and r is some residual vector, not containing the support 
of any nonzero codeword in its support, that remains after subtracting the codeword vectors , . . . , c( fc ) from 
p. Either r is the all-zeros vector, or r is a vector comprising of or even entries only. 

This lemma describes a particular composition of a pseudocodeword p. Note that the above result does not 
claim that p is reducible even though the vector p can be written as a sum of codeword vectors c^ 1 ), . . . , c^ k \ 
and r. Since r need not be a pseudocodeword, it is not necessary that p be reducible structurally as a sum of 
codewords and/or pseudocodewords (as in Definition 12.7b - It is also worth noting that the decomposition of a 
pseudocodeword, even that of an irreducible pseudocodeword, is not unique. 

Example 5.1: For representation B of the [7, 4, 3] Hamming code as shown in Figure[T9lin Section 6, label the 
vertices clockwise from the top as ^1,^2,^3,^4, ^5, Vq, and vj. The vector p = (p\, . . . ,p n ) = (1, 2, 1, 1, 1, 0, 2) 
is an irreducible pseudocodeword and may be decomposed as p = (1,0,1,0,0,0,1) + (0,0,0,1,1,0,1) + 
(0, 2, 0, 0, 0, 0, 0) and also as p = (1, 0, 1, 1, 1, 0, 0) + (0, 2, 0, 0, 0, 0, 2). In each of these decompositions, each 
vector in the sum is a codeword except for the last vector which is the residual vector r. 

Theorem 5.1: Let p = (p\, . . . ,p n ) be a pseudocodeword. If there is a decomposition of p as in Lemma [5721 
such that r = 0, then p is a good pseudocodeword as in Definition 12.81 
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Fig. 16. A minimal stopping set with property G. 

Theorem 5.2: The following are sufficient conditions for a pseudocodeword p = (pi,... ,p n ) to be bad, as 
in Definition 12.91 

1) w BSC / AWGN (p)<d min . 

2) | supp(p)| < d min . 

3) If p is an irreducible nc-pseudocodeword and | supp(p) | > I + 1, where £ is the number of distinct 
codewords whose support is contained in supp(p). 

Intuitively, it makes sense for good pseudocodewords, i.e., those pseudocodewords that are not problematic 
for iterative decoding, to have a weight larger than the minimum distance of the code, d m - m . However, we note 
that bad pseudocodewords can also have weight larger than d m i n . 

Definition 5.1: A stopping set S has property G if S contains at least one pair of variable nodes u and v 
that are not connected by any path that traverses only via degree two check nodes in the subgraph G\ s of G 
induced by S in G. 

Example 5.2: In Figure [10] in Section 4, the set {^1,^2^4} is a minimal stopping set and does not have 
property G, whereas the set {v\, V3, U4, v§, Vq, W7, V10, «u, «i2? ^13} is not minimal but has property G. The 
graph in Figure [16] is a minimal stopping set that has property Q. The graph in Example 14.11 has no minimal 
stopping sets with property Q, and all stopping sets have size at least the minimum distance d mm . 

Lemma 5.3: Let S be a stopping set in G. Let ts denote the largest component an irreducible pseudocode- 
word with support S may have in G. If S is a minimal stopping set and does not have property G, then a 
pseudocodeword with support S has maximal component 1 or 2. That is, ts = 1 or 2. 

Subgraphs of the LDPC constraint graph may also give rise to bad pseudocodewords, as indicated below. 

Definition 5.2: A variable node v in an LDPC constraint graph G is said to be problematic if there is a 
stopping set S containing v that is not minimal but nevertheless has no proper stopping set S' C S for which 

v e S'. 

Observe that all graphs in the examples of Section 4 have problematic nodes and conditions 1 and 3 in 
Theorem 15.21 are met in Examples 14.21 and 14.31 The problematic nodes are the variable nodes in the inner 
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Fig. 17. A non-minimal stopping set as in case 2(b)(ii) of Theorem 15.31 

ring in Example 14. 1 1 the nodes V7,v% in Example 14.21 and v% in Example 14.31 Note that if a graph G has a 
problematic node, then G necessarily contains a stopping set with property 0. 

The following result classifies bad nc-pseudocodewords, with respect to the AWGN channel, using the graph 
structure of the underlying pseudocodeword supports, which, by Lemma 12.11 are stopping sets in the LDPC 
constraint graph. 

Theorem 5.3: Let G be an LDPC constraint graph representing an LDPC code C, and let S be a stopping 
set in G. Then, the following hold: 

1) If there is no non-zero codeword in C whose support is contained in S, then all nc-pseudocodewords of 
G, having support equal to S, are bad as in Definition 12.91 Moreover, there exists a bad pseudocodeword 
in G with support equal to S. 

2) If there is at least one codeword c whose support is contained in S, then we have the following cases: 

(a) if S is minimal, 

(i) there exists a nc -pseudocodeword p with support equal to S iff S has property 6. 

(ii) all nc-pseudocodewords with support equal to S are bad. 

(b) if S is not minimal, 

(i) and S contains a problematic node v such that v ^ S' for any proper stopping sej^l S' C 
S, then there exists a bad pseudocodeword p with support S. Moreover, any irreducible nc- 
pseudocodeword p with support S is bad. 

(ii) and S does not contain any problematic nodes, then every variable node in S is contained in a 
minimal stopping set within S. Moreover, there exists a bad nc-pseudocodeword with support 
S iff either one of these minimal stopping sets is not the support of any non-zero codeword in 
C or one of these minimal stopping sets has property 0. 

The graph in Figure QjJ is an example of case 2(b)(ii) in Theorem [53] Note that the stopping set in the figure 
is a disjoint union of two codeword supports and therefore, there are no irreducible nc-pseudocodewords. 

The graph in Figure [18] is an example of case 2(a). The graph has property and therefore has nc- 
pseudocodewords, all of which are bad. 

2 A proper stopping set S' of S is a non-empty stopping set that is a strict subset of S. 
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Fig. 18. A minimal stopping set as in case 2(a) of Theorem 15.31 



A. Remarks on the weight vector and channels 

In [6], Frey et. al show that the max-product iterative decoder (equivalently, the MS iterative decoder) will 
always converge to an irreducible pseudocodeword (as in Definition 12.71 ) on the AWGN channel. However, 
their result does not explicitly show that for a given irreducible pseudocodeword p, there is a weight vector 
w such that the cost pw T is the smallest among all possible pseudocodewords. In the previous subsection, 
we have given sufficient conditions under which such a weight vector can explicitly be found for certain 
irreducible pseudocodewords. We believe, however, that finding such a weight vector w for any irreducible 
pseudocodeword p may not always be possible. In particular, we state the following definitions and results. 

Definition 5.3: A truncated AWGN channel, parameterized by L and denoted by TAWGN(L), is an AWGN 
channel whose output log-likelihood ratios corresponding to the received values from the channel are truncated, 
or limited, to the interval [— L, L]. 

In light of [15], [16], we believe that there are fewer problematic pseudocodewords on the BSC than on the 
truncated AWGN channel or the AWGN channel. 

Definition 5.4: For an LDPC constraint graph G that defines an LDPC code C, let Pawgn(G) rje tne set 
of lift-realizable pseudocodewords of G where for each pseudocodeword p in the set, there exists a weight 
vector w such that the cost pw T on the AWGN channel is the smallest among all possible lift-realizable 
pseudocodewords in G. 

Let PE sc (G) and Ptawgn(L)(G) be defined analogously for the BSC and the truncated AWGN channel, 
respectively. Then, we have the following result: 

Theorem 5.4: For an LDPC constraint graph G, and L > 1, we have 



The above result says that there may be fewer problematic irreducible pseudocodewords for the BSC than 
over the TAWGN(L) channel and the AWGN channel. In other words, the above result implies that MS iterative 
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Fig. 19. Three different representations of the [7,4,3] Hamming code. 

decoding may be more accurate for the BSC than over the AWGN channel. Thus, quantizing or truncating the 
received information from the channel to a smaller interval before performing MS iterative decoding may be 
beneficial. (Note that while the above result considers all possible weight vectors that can occur for a given 
channel, it does not take into account the probability distribution of weight vectors for the different channels, 
which is essential when comparing the performance of MS decoding across different channels.) Since the 
set of lift-realizable pseudocodewords for MS iterative decoding is the set of pseudocodewords for linear- 
programming (LP) decoding (see Section 2), the same analogy carries over to LP decoding as well. Indeed, 
at high enough signal to noise ratios, the above observation has been shown true for the case of LP decoding 
in [15] and more recently in [16]. 

VI. Graph Representations and Weight Distribution 

In this section, we examine different representations of individual LDPC codes and analyze the weight 
distribution of lift-realizable pseudocodewords in each representation and how it affects the performance of 
the MS iterative decoder. We use the classical [7,4,3] and [15, 11,3] Hamming codes as examples. 

Figure [19] shows three different graph representations of the [7, 4, 3] Hamming code. We will call the 
representations A, B, and C, and moreover, for convenience, also refer to the graphs in the three respective 
representations as A, B, and C. The graph A is based on the systematic parity check matrix representation 
of the [7, 4, 3] Hamming code and hence, contains three degree one variable nodes, whereas the graph B 
has no degree one nodes and is more structured (it results in a circulant parity check matrix) and contains 4 
redundant check equations compared to A, which has none, and C, which has one. In particular, A and C are 
subgraphs of B, with the same set of variable nodes. Thus, the set of lift-realizable pseudocodewords of B is 
contained in the set of lift-realizable pseudocodewords of A and C, individually. Hence, B has fewer number 
of lift-realizable pseudocodewords than A or C. In particular, we state the following result: 



Theorem 6.1: The number of lift-realizable pseudocodewords in an LDPC graph G can only reduce with 
the addition of redundant check nodes to G. 
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The proof is obvious since with the introduction of new check nodes in the graph, some previously valid 
pseudocodewords may not satisfy the new set of inequality constraints imposed by the new check nodes. (Recall 
that at a check node c having variable node neighbors Vi k , a pseudocodeword p = (p±, . . . ,p n ), must 

satisfy the following inequalities pi. < Ylh^j h=i kPih? f° r 3 ' = 1; • • • > & (see equation ©)■) However, the 
set of valid codewords in the graph remains the same, since we are introducing only redundant (or, linearly 
dependent) check nodes. Thus, a graph with more check nodes can only have fewer number of lift-realizable 
pseudocodewords and possibly a better pseudocodeword-weight distribution. 

If we add all possible redundant check nodes to the graph, which, we note, is an exponential number 
in the number of linearly dependent rows of the parity check matrix of the code, then the resulting graph 
would have the smallest number of lift-realizable pseudocodewords among all possible representations of the 
code. If this graph does not have any bad nc -pseudocodewords (both lift-realizable ones and those arising on 
the computation tree) then the performance obtained with iterative decoding is the same as the optimal ML 
performance. 

Remark 6.1: Theorem 16.11 considers only the set of lift-realizable pseudocodewords of a Tanner graph. On 
adding redundant check nodes to a Tanner graph, the shape of the computation tree is altered and thus, it is 
possible that some new pseudocodewords arise in the altered computation tree, which can possibly have an 
adverse effect on iterative decoding. The [4,1,4] repetition code example from Section 2.C illustrates this. 
Iterative decoding is optimal on the single cycle representation of this code. However, on adding a degree 
four redundant check node, the iterative decoding performance deteriorates due to the introduction of bad 
pseudocodewords to the altered computation tree. (See Figure [7]) (The set of lift-realizable pseudocodewords 
however remains the same for the new graph with redundant check nodes as for the original graph.) 

Returning to the Hamming code example, graph B can be obtained by adding edges to either A or C, 
and thus, B has more cycles than A or C. The distribution of the weights of the irreducible lift-realizable 
pseudocodewords for the three graphs A, B, and C is shown§ in Figure [20] (The distribution considers all 
irreducible pseudocodewords in the graph, since irreducible pseudocodewords may potentially prevent the MS 
decoder to converge to any valid codeword [6].) Although, all three graphs have a pseudocodeword of weight 
three0, Figure [20] shows that B has most of its lift-realizable pseudocodewords of high weight, whereas C, 
and more particularly, A, have more low-weight lift-realizable pseudocodewords. The corresponding weight 
distributions over the BEC and the BSC channels are shown in Figure [2T] B has a better weight distribution 

3 The plots considered all pseudocodewords in the three graphs that had a maximum component value of at most 3. Hence, for 
each codeword c, 2c and 3c are also counted in the histogram, and each has weight at least d m in- However, each irreducible nc- 
pseudocodeword p is counted only once, as p contains at least one entry greater than 1, and any nonzero multiple of p would have a 
component greater than 3. The t-value (see Section 5) is 3 for the graphs A, B, and C of the [7, 4, 3] Hamming code. 

4 Note that this pseudocodeword is a valid codeword in the graph and is thus a good pseudocodeword for iterative decoding. 
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Fig. 20. Pseudocodeword-weight (AWGN) distribution of representations A,B,C of the [7,4,3] Hamming code. 
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Fig. 21. Pseudocodeword-weight distribution (BEC and BSC channels) of representations A, B, C of the [7,4,3] Hamming code. 



than A and C over these channels as well. 



The performance of MS iterative decoding of A, B, and C on the BIAWGNC with signal to noise ratio 
Eb/N a is shown in Figures |22l [23] and |24l respectively. (The maximum number of decoding iterations was 
fixed at 100.) The performance plots show both the bit error rate and the frame error rate, and further, they also 
distinguish between undetected decoding errors, that are caused due to the decoder converging to an incorrect 
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but valid codeword, and detected errors, that are caused due to the decoder failing to converge to any valid 
codeword within the maximum specified number of decoding iterations, 100 in this case. The detected errors 
can be attributed to the decoder trying to converge to an nc-pseudocodeword rather than to any valid codeword. 

Representation A has a significant detected error rate, whereas representation B shows no presence of 
detected errors at all. All errors in decoding B were due to the decoder converging to a wrong codeword. 
(We note that an optimal ML decoder would yield a performance closest to that of the iterative decoder 
on representation B.) This is interesting since the graph B is obtained by adding 4 redundant check nodes 
to the graph A. The addition of these 4 redundant check nodes to the graph removes most of the low- 
weight nc-pseudocodewords that were present in A. (We note here that representation B includes all possible 
redundant parity-check equations there are for the [7,4,3] Hamming code.) Representation C has fewer number 
of pseudocodewords compared to A. However, the set of irreducible pseudocodewords of C is not a subset 
of the set of irreducible pseudocodewords of A. The performance of iterative decoding on representation C 
indicates a small fraction of detected errors. 

Figure [25] compares the performance of min-sum decoding on the three representations. Clearly, B, having 
the best pseudocodeword weight distribution among the three representations, yields the best performance with 
MS decoding, with performance almost matching that of the optimal ML decoder. 

Figure [26] shows the performance of the three different representations over the BSC channel with MS 
iterative decoding. Since there are only 2 7 = 128 different error patterns, the performance of MS decoding for 
each error pattern was determined and the average number of output errors were computed. Representations A 
and C failed to correct any non-zero error pattern whereas representation B corrected all one-bit error patterns. 
The performance of MS decoding using representation B was identical to the performance of the ML decoder 
and the MS decoder always converged to the ML codeword with representation B. This goes to show that 
representation B is in fact the optimal representation for the BSC channel. 

Similarly, we also analyzed three different representations of the [15, 11,3] Hamming code. Representation 
A has its parity check matrix in the standard systematic form and thus, the corresponding Tanner graph has 
4 variable nodes of degree one. Representation B includes all possible redundant parity check equations of 
representation A and has the best pseudocodeword-weight distribution. Representation C includes up to order- 
two redundant parity check equations from the parity check matrix of representation A, meaning, the parity 
check matrix of representation C contained all linear combinations of every pair of rows in the parity check 
matrix of representation A. Thus, its (lift-realizable) pseudocodeword-weight distribution is superior to that of 
A but inferior to that of B. (See Figure 1301 ) 

The analogous performance of MS iterative decoding of representations A, B, and C of the [15,11,3] 
Hamming code on a BIAWGNC with signal to noise ratio E\,/N is shown in Figures [27] [28] and [29] 
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[7,4,3] Hamming Code: Representation A 



[7,4,3] Hamming Code: Representation B 
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Fig. 22. Representation A. 
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Fig. 23. Representation B. 
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Fig. 24. Representation C. Fig. 25. Comparison between representations. 

Performance of the [7,4,3] Hamming code with min-sum iterative decoding over the BIAWGNC. 



respectively. (The maximum number of decoding iterations was fixed at 100.) We observe similar trends 
in the performance curves as in the previous example. A shows a prominent detected error rate, whereas B 
and C show no presence of detected errors at all. The results suggest that merely adding order two redundant 
check nodes to the graph of A is sufficient to remove most of the low-weight pseudocodewords. 



Inferring from the empirical results of this section, we comment that LDPC codes that have structure and 
redundant check nodes, for example, the class of LDPC codes obtained from finite geometries [17], are likely to 
have fewer number of low-weight pseudocodewords in comparison to other randomly constructed LDPC graphs 
of comparable parameters. Despite the presence of a large number of short cycles (i.e., 4-cycles and 6-cycles), 
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Performance of [7,4,3] Hamming code representations with min-sum decoding 
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Fig. 26. Performance of the [7,4,3] Hamming code representations with MS decoding over the BSC. 

the class of LDPC codes in [17] perform very well with iterative decoding. It is worth investigating how the 
set of pseudocodewords among existing LDPC constructions can be improved, either by adding redundancy or 
modifying the Tanner graphs, so that the number of (bad) pseudocodewords, both lift-realizable ones as well 
as those occurring on the computation tree, is lowered. 

VII. Conclusions 

This paper analyzed pseudocodewords of Tanner graphs, with the focus on the structure, bounds on the 
minimum pseudocodeword weight, and iterative decoding performance. It would be worthwhile to relate the 
results in Section 6 to the stopping redundancy, as introduced in [18]. Since this paper primarily dealt with lift- 
realizable pseudocodewords, the results presented are also applicable in the analysis of LP decoding. We hope 
the insights gained from this paper will aid in the design of LDPC codes with good minimum pseudocodeword 
weights. 
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Appendix 

Proof of Lemma 127} Consider the subgraph Gig of G induced by the set of vertices S = supp(p). Observe 
that every cloud of check nodes in the corresponding cover of G\$ is connected to either none or at least two 
of the variable clouds in the support of p. If this were not the case, then there would be a cloud of check 
nodes in the cover with at least one check node in that cloud connected to exactly one variable node of bit 
value one, thus, leaving the check node constraint unsatisfied. Therefore, the corresponding variable nodes in 
the base graph G satisfy the conditions of a stopping set. ■ 

Proof of Claim 12.71 Let G represent the constraint graph of an LDPC code C. Suppose G is a tree, then 
clearly, any pseudocodeword of G can be expressed as a linear combination of codewords of G. Hence, suppose 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



33 



G is not a tree, and suppose all check nodes in G are of degree two. Then the computation tree contains only 
check nodes of degree two and hence, for a valid assignment on the computation tree, the value of any child 
variable node vi on the computation tree that stems from a parent check node u is the same as the value of the 
variable node V2 which is the parent node of it. Thus, the only local codeword configurations at each check 
node is the all-ones configurations when the root node of the tree is assigned the value one. Hence, the only 
valid solutions on the computation tree correspond to the all ones vector and the all zeros vector - which are 
valid codewords in C. 

Conversely, suppose G is not a tree and suppose there is a check node u of degree k in G. Let v\, V2, ■ ■ ■ , Vf. 
be the variable node neighbors of u. Enumerate the computation tree rooted at v\ for a sufficient depth such 
that the node u appears several times in the tree and also as a node in the final check node layer of the tree. 
Then there are several possible valid assignments in the computation tree, where the values assigned to the 
leaf nodes that stem from u yield a solution that is not a valid codeword in G. Thus, G contains irreducible 
nc-pseudocodewords on its computation tree. ■ 

Proof of Claim \2.2\ Let G represent the constraint graph of an LDPC code C. Suppose G is a tree, then 
clearly, any pseudocodeword of G can be expressed as a linear combination of codewords of G. Hence, suppose 
G is not a tree, and between every pair of variable nodes in G there is a path that contains only degree two 
check nodes in G. Then G contains only lift-realizable pseudocodewords of the form {k, k, ... , k), where k 
is a positive integer. Hence, the only irreducible pseudocodewords in G are the all-zeros vector (0, 0, ... , 0) 
and either the all-ones vector (1,1,..., 1) or the all-twos vector (2, 2, ... , 2) (if the all-ones vector is not a 
codeword in G). ■ 

Proof of Theorem \3.1\ 

Case: | odd. At a single constraint node, the following inequality holds (see equation [[J: 



Applying this inequality in the LDPC constraint graph enumerated as a tree with the root node corresponding 
to the dominant pseudocodeword component a\ , we have 





where Lq corresponds to variable nodes in the first level (level 0) of the tree. Similarly, we have 
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Fig. 31. A single constraint node and a d-left regular (bipartite) tree enumerated from an arbitrary variable node. 



and so on, until, 

d(d — l) i *~ai < a j 

Since the LDPC graph has girth g, the variable nodes up to level Ls^efl are all distinct. The above inequalities 

4 

yield: 

l + d + d(d-l)H hd(d-l) 2 ^ «i < ^ «i<^ai 

ie{l}UL U...Lg-6 aZZ 

Without loss of generality, let us assume, a\ > ai > • • • > a e to be the e dominant components in p. That 



is, ai + a.2 H h a e > 



Since, each is at most ai, we have Xa=i a « ^ ea i- This implies that 



eai > J^ai > ^ T~ 



1 + d + d(d - 1) + • • • + d(d - 1) V 



e > 



2 ~ 2 
[l + d + d(d- !) + ••• + 



Since wbsc = 2e, the result follows. (The case when | is even is treated similarly.) 



AWGA/ case: Let x 



[lH-d+d(d-l)H hd(d-l) J 



-. Since, 



l + d + <f(d- 1) + ••• + £*(<* -1) 



a i 



5 Note that Li refers to the level for which the exponent of the (d — 1) term is i. 



• • • a n . 
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we can write Ya=i a i = ( x + 2/) a i> where y is some non-negative quantity. Suppose a\ H \-a e = a e+ \ + 

Then, since we have £? =1 af < 2 £-=i a i < 2 "i(Ei=i a i) = 2 ( x + we § et ' 

^ * ^ T) O _ 



^> „, -T L 2 =2(x + y)>2x. 



4(x + yfaj 
2{x + y)a\ 

(The case ai + • • • + a e > a e +i + • • • a n is treated similarly.) 



Proof of Theorem 13.21 As in the proof of Theorem 13.11 where we note that for a single constraint with 
neighbors having pseudocodeword components a\, . . . , aik, we have the following relation (for a^, i = 1, . . . , k): 

k 

(ek)ai < a j- 
i=i 

The result follows by applying this inequality at every constraint node as in the proof of Theorem 13.11 ■ 

Proof of Lemma 1X7} Let p = (ai, . . . ,a n ) be a pseudocodeword of G, and without loss of generality, let 
cti > a2 > ■ ■ ■ > a n . To establish the inequality for the AWGN channel, we need to show 



(Ei=i a i) > Ei=i o< _ Ei=i 



Ya=1 a i max « "1 

Since J2i a i — a i + 0102 + • • • + «ia„ = a\(J2i trns implies 

(E?=i a i) 2 > (EIU^ > ET=i oh 



22i=i a t «i Li=i on ai 
Hence, u; AiyG7V (p) > w max -frac(p)- 

To establish the bound for the BSC, let e be the smallest number such that Ylt=i a * — ^'l* ~ • First 
suppose Ei=i a * = Er= e +i«i- Then ^ B5C (p) = 2e. Moreover, u> ma x-frac(p) = = 2 ^T 1 "' - Each 



a, < ai w m a X -frac(p) < ^ = 2e = w i<AG (p). Now suppose E- =1 aj > EiLe+l"*- Then ' for some 



5 > 0, we have £? =1 Oj = £? =e+1 «< + We have «W-frac(p) = = ^ ■ Note that 

2eai 



Ei=l «i + E?=e+1 «i < 2 ELl < ( 2e )«l" ThUS ' ^max-frac(p) < ^ = 2e. ■ 

Proof of Corollary \3.4\ Follows from Lemma 13.11 and Theorem 13.31 ■ 

Proof of Theorem 13.51 In the polytope representation introduced by Koetter and Vontobel in [5], irreducible 
pseudocodewords correspond to the edges of the fundamental cone, which can be described by {x [ x\ > 0, i = 
1, . . . , 11} and {x I X{ < Yli>£N(j)\{i} Xi '} ^ or a ^ cnec k nodes j and all variable nodes i in N(j). Consider the 
polytope that is the intersection of the fundamental cone with the hyperplane x\ + ■ ■ ■ + x n = 1. The vertices 
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of this polytope have a one-to-one correspondence with the edges of the fundamental cone. Let v be a fixed 
vertex of this new polytope. Then v satisfies at least n — 1 of the above inequalities with equality. Together with 
the hyperplane equality, v meets at least n inequalities with equality. The resulting system of linear equations 
contains only integers as coefficients. By Cramer's rule, the vertex v must have rational coordinates. Taking 
the least common multiple of all denominators of all coordinates of all vertices gives an upper bound on t. 
Therefore, t is finite. ■ 



Proof of Theorem \3.6[ Let m be the minimum degree lift needed to realize the given pseudocodeword p. 
Then, in a degree m lift graph G that realizes p, the maximum number of active check nodes in any check 
cloud is at most m. A check cloud u is connected to J2ieN(u) Pi act i ve variable nodes from the variable clouds 
adjoining check cloud u. (Note that N(u) represents all the variable clouds adjoining u.) Since every active 
check node in any check cloud has at least two (an even number) active variable nodes connected to it, we 
have that 2m < fnax u ^ i&N ^pi. This quantity can be upper-bounded by td^ since pi < t, for all i, and 
\N(u)\ < d+, for all u. ■ 

Proof of Lemma \3.2\ (a) AWGN case: Let be the number of pj's that are equal to k, for k = 1, . . . , t. 
The pseudocodeword weight is then equal to: 



w AWGN {p) 



(m + 2n 2 + • • • + tn t ) 2 



(m + 2 2 n 2 + • • • + t 2 n t )' 

Now, we have to find a number r such that w AWGN (p) > r|supp(p)|. Note however, that |supp(p)| = 
n\ + n 2 + • • • + n t . This implies that for an appropriate choice of r, we have 

(m + 2n 2 + • • • + tn t ) 2 A . 2 2 A A (i 2 + j 2 )r-2ij 

— r > r(ni + •••71+) or > i n,- > > > n^n,- (*) 

(ni + 2 2 n 2 + • • • + t 2 n t ) ~ v ; ^ 1 ~ ^ ^ 1-r 

i=l i=l j=i+l 

Note that r < 1 in the above. Clearly, if we set r to be the minimum over all 1 < i < j < n such that 
(i 2 +j 2 )r < 2ij, then it can be verified that this choice of r will ensure that (*) is true. This implies r = 
(for i = l,j = t). 

However, observe that left-hand-side (LHS) in (*) can be written as the following LHS: 
1 rr/'2 2, -2 2^rf {i 2 +j 2 )r-2ij 

i=l j=i+l i=l j=i+l 

Now, using the inequality a 2 + b 2 > 2ab, r can be taken as the minimum over all 1 < i < j < t such that 
jhiii 2 ™! + i 2jl j) > r( ' 24 1 J -r~ 2 ' : ' nin i- The smallest value of r for which this inequality holds for all 1 < i < j < t 

(l+t 2 )(f.-l)+2t' 



is given by r = (1 , 2 3 t ._, , 2f , thereby proving the lemma in the AWGN case. 



BSC case: 



Since w BSC (p) > ■w max -frac(p), we have from Lemma ED that w BSC (p) > Pl + a ^+f" > |sup t p(p)l . (Note 
that the p^s are non-negative integers.) Therefore, w BSC \p) > j \ supp(p)|. ■ 
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Proof of Theorem \3.8\ Let p be a good pseudocodeword. This means that if for any weight vector w we 
have cw T > for all 7^ c G C, then, pw T > 0. Let us now consider the BSC and the AWGN cases 
separately. 

BSC case: Suppose at most ^p errors occur in channel. Then, the corresponding weight vector w will have 
^p or fewer components equal to —1 and the remaining components equal to +1. This implies that the cost 
of any / c 6 C (i.e., cw T ) is at least since there are at least d m - m l's in support of any / c £ C. Since 
p is a good pseudocodeword, it must also have positive cost, i.e. pw T > 0. Let us assume that the —l's occur 
in the dominant ^^p positions of p, and without loss of generality, assume p\ > P2 > . . . > p n - (Therefore, 

w = (-1, -1, . . . , -1, +1, +1, . . . , +1).) Positive cost of p implies p x ^ \-Pd min /2 < P(<W/2)+i H ^Vn- 

So we have e > ^p, where e is as defined in the pseudocodeword weight of p for the BSC. The result follows. 

AWGN case: Without loss of generality, let p\ be dominant component of p. Set the weight vector w = (1 — 
rfmin, L • • • , 1)- Then it can be verified that cw T > for any / c 6 C. Since p is a good pseudocodeword, 
this implies p also must have positive cost. Cost of p is (1 — <i m in)pi +P2 + • • • +Pn > d m i n < Pl+ p ^ +p " ■ 
Note that the right-hand-side (RHS) is w m a X -frac(p); hence, the result follows from Lemma I37T1 

Now let us consider p to be a bad pseudocodeword. From Lemma l27Tl we have | supp(p) | > s m - m . Therefore, 
^max-frac(p) > ^p 1 (since p\ = t is the maximum component of p), and hence, the result follows by 
Lemmas 13.11 and 13.21 ■ 

Proof of Lemma 15.71 Consider a graph H having a single check node which is connected to variable nodes 
vi. ... , Vk- Suppose b = (61, . . . , 6fc) is a pseudocodeword in H, then b corresponds to a codeword in a lift 
H of H. Every check node in H is connected to an even number of variable nodes that are assigned value 1, 
and further, each variable node is connected to exactly one check node in the check cloud. Since the number 
of variable nodes that are assigned value 1 is equal to the sum of the 6j's, we have ^ hi = mod 2. 

Let G be the corresponding lift of G wherein p forms a valid codeword. Then each check node in G is 
connected to an even number of variable nodes that are assigned value 1. From the above observation, if nodes 
Vi k participate in the check node U{ in G, then Pi t + ■ ■ ■ + Pi k = mod 2. Let Xi = pi mod 2, for 
i = 1, . . . ,n (n being the number of variable nodes, i.e., the block length of C, in G). Then, at every check 
node Ui, we have X{ ± + • • • + Xi k =0 mod 2. Since x = (xi, . . . , x n ) = p mod 2 is a binary vector satisfying 
all checks, it is a codeword in C. ■ 

Proof of Lemma 1372} Suppose c G C is in the support of p, then form p' = p — c. If p' contains a codeword 
in its support, then repeat the above step on p'. Subtracting codewords from the pseudocodeword vector in 
this manner will lead to a decomposition of the vector p as stated. Observe that the residual vector r contains 
no codeword in its support. 

From Lemma 157X1 x = p mod 2 is a codeword in C. Since p = + ■ • • + r, we have x = (c^ 1 ) + • • • + 
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c( fc )) mod 2 + r mod 2. But since x E C, this implies r mod 2 G C. However, since r contains no codeword 
in its support, r mod 2 must be the all-zero codeword. Thus, r contains only even (possibly 0) components. 
■ 

Proof of Theorem 15.71 Let p be a pseudocodeword of a code C, and suppose p may be decomposed as 
p = c^ 1 ' + c^ 2 ) + • ■ ■ + c^ k \ where {cW}J =1 is a set of not necessarily distinct codewords. Suppose p is bad. 
Then there is a weight vector w such that pw T < but for all codewords c € C, cw T > 0. Having pw T < 
implies that c^^w T + c^ 2 ^w T + • • • + c( fc )w T = —x, for some positive real value x. So there is at least one i 
for which c^w T < 0, which is a contradiction. Therefore, p is a good pseudocodeword. ■ 

Proof of Theorem I5.2| - Let M be an arbitrarily large finite positive integer. 

1) If yjBSC/AWGN ^p-j < cf m in, then p is a bad pseudocodeword by Theorem 13.81 

2) If | supp(p) | < rfmin, then there is no codeword in the support of p, by Lemma 15.11 Let w = 
(u>i, . . . , w n ), be a weight vector where for i = 1, 2, . . . , n, 

{-1 if V{ € supp(p), 
M if Vi £ supp(p) 

Then pw T < and for all codewords c € C, cw T > 0. 

3) Suppose p is a irreducible nc -pseudocodeword. Without loss of generality, assume p = (pi,P2, ■ ■ ■ ,Ps> 0, 0, . 
i.e., the first s positions of p are non-zero and the rest are zero. Suppose p contains £ distinct codewords 
c^'jc' 2 ', . . . , in its support. Then if s > I + 1, we define a weight wector w = (wi, W2, • • • , w n ) 
as follows. Let W{ = M for i £ supp(p). Solve for w\, u>2, ■ ■ ■ , w s from the following system of linear 
equations: 

C (D W T = +1) 

c Ww T = +1, 
pw T = p x wi + p 2 w 2 H h p s w s = -2, 

The above system of equations involves s unknowns w\, w 2 , ■ ■ ■ , w s and there are £ + 1 equations. 
Hence, as long as s > £ + 1, we have a solution for the WiS. (Note that for the case s = £ + 1, when 
the {£ + 1) x (£ + 1) matrix containing the first s components of the codewords c^, . . . and the 
pseudocodeword p as its rows, has a non-zero determinant, there is exactly one solution for the Wi's and 
when this determinant is zero, there is either zero or more than one solution.) 

Thus, there exists a weight vector w = (w\, . . . , w s , M, . . . , M) such that pw T < and cw T > for 
all codewords c in the code. This proves that p is a bad pseudocodeword. 
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Proof of Theorem \5.3[ Let M be an arbitrarily large finite positive integer. 

1) Let S be a stopping set. Suppose there are no non-zero codewords whose support is contained in 

S. The pseudocodeword p with component value 2 in the positions of S, and elsewhere is then a bad 
pseudocodeword on the AWGN channel, which may be seen by the weight vector w a = (w\, . . . , w n ), 
where for i = 1, 2, . . . , n, 

f -1 if i € S, 

Wi= < 

I M if * S 

In addition, since all nonzero components have the same value, the weight of p on the BSC and AWGN 
channel is \S\. 

Suppose now that p is a nc -pseudocodeword with support S. Then the weight vector w a again shows 
that p is bad, i.e., pw5f < and c'w^ > for all c' G C. 

2) Suppose there is at least one non-zero codeword c whose support is in S. 

(a) Assume 5 is a minimal stopping set. Then this means that c is the only non-zero codeword whose 
support is in S and supp(c) = S. 

(i) Suppose S has property 0, then we can divide the variable nodes in S into disjoint equivalence classes 
such that the nodes belonging to each class are connected pairwise by a path traversing only via degree 
two check nodes in G\ s . Consider the pseudocodeword p having component value 3 in the positions 
corresponding to all nodes of one equivalence class, component value 1 for the remaining positions of 
S, and component value elsewhere. (It can be quickly verified that this is indeed a pseudocodeword 
by considering the subgraph induced by S. In this subgraph, variable nodes from different equivalence 
classes will be connected together at check nodes of degree greater than two. Since there is a non-zero 
codeword with support in S, all such check nodes have even degree. This implies that the pseudocodeword 
inequalities of the form pi < J2j^i Pj ( see equation []} at each check node is satisfied, implying that the 
chosen p is a pseudocodeword.) Let r = p — c, and let i denote the index of the first non-zero component 
of r and i* denote the index of the first non-zero component in supp(p) — supp(r). The weight vector 
w b = (wi, ■ ■ ■ , w n ), where for i = 1, 2, . . . , n, 



-1 


if i = i, 


+1 


i = i* 


M 


Hi is 





otherwise 



ensures that p is bad as in Definition |2.9[ and it is easy to show that the weight of p on the AWGN 
channel is strictly less than | jS' | . 
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Conversely, suppose S does not have property 6. Then every pair of variable nodes in S is connected by 
a path in G\ s that contains only degree two check nodes. This means that any pseudocodeword p with 
support S must have all its components in S of the same value. Therefore, the only pseudocodewords 
with support S that arise have the form p = kc, for some positive integer k. (By Theorem 15.11 these 
are good pseudocodewords.) Hence, there exists no bad pseudocodewords with support S. 

(ii) Let p be a nc -pseudocodeword with support S. If S contains a codeword c in its support, then 
since S is minimal supp(c) = S. Let k denote the number of times c occurs in the decomposition (as 
in Lemma 15.21 ) of p. That is, p = kc + r. Note that r is non-zero since p is a nc-pseudocodeword. 
Let i denote an index of the maximal component of r, and let i* denote the index of the first nonzero 
component in supp(p) — supp(r). The weight vector w^, defined above, again ensures that p is bad. 

(b) Suppose S is not a minimal stopping set and there is at least one non-zero codeword c whose 
support is in S. 

(i) Suppose S contains a problematic node v. By definition, assume that S is the only stopping set among 
all stopping sets in S that contains v. Define a set S v as 

S v := {u G S\ there is a path from u to v containing only degree two check nodes in G| s } 

Then, the pseudocodeword p that has component value 4 on all nodes in S v , component value 2 on all 
nodes in S — S v and component value everywhere else is a valid pseudocodeword. (It can be quickly 
verified that this is indeed a pseudocodeword by considering all the check nodes in the graph induced 
by S in G. Any check node that is in the path from some u G S v to v is either of degree two, in which 
case the inequality in equation £0) is satisfied, or is of degree greater than two, in which case the check 
node is possibly connected to a variable node in S — S v and to at least two variable nodes in S v . The 
choice of the components guarantee that the pseudocodeword inequality in equation (Q]) is still satisfied 
at this check node. For any other check node in G\ g , the degree is at least two, and it is easy to verify 
that the inequality in equation (GQ) is satisfied.) Let i* be the index of the variable node v in G, and let 
i' be the index of some variable node in S — S v . Then, the weight vector w c = (w\, . . . , w n ), where for 
i = 1,2, ...,n, 



Wi= < 



-1 


if i = i*, 


+1 


i = if 


M 


if % £ S 





otherwise 
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ensures that pwj < and c'wJT > for all non-zero codewords c' in C. Hence, p is a bad pseudocode- 
word with support S. This shows the existence of a bad pseudocodeword on S. 

Suppose now that p' is some irreducible nc -pseudocodeword with support S. If there is a non-zero 
codeword c such that c has support in S and contains v in its support, then since v is a problematic 
node in S, v cannot lie in a smaller stopping set in S. This means that support of c is equal to S. We 
will show that p' is a bad pseudocodeword by constructing a suitable weight vector wj. Let i' be the 
index of some variable node in S — S v . Then we can define a weight vector w<j — (wi, . . . , w n ), where 
for i = 1, 2, . . . , n, 



Wi = < 



-1 


if i = i*, 


+1 


i = i' 


M 


if i<£ S 





otherwise 



(Note that since p' is a irreducible nc -pseudocodeword and contains v in its support, p\, > p-,.) This 
weight vector ensures that p'wj < and c'wj > for all non-zero codewords c' G C. Thus, p' is a 
bad pseudocodeword. 

If there is no non-zero codeword c such that c has support in S and also contains v in its support. Then, 
the weight vector w c = (w\, . . . , w n ), where for i = 1, 2, . . . , n, 

{-1 if i = i*, 
M if i <£ S 
otherwise 

ensures that p'w<T < and c'wj > for all non-zero codewords c' G C. Thus, p' is a bad pseudocode- 
word. This proves that any irreducible nc-pseudocodeword with support S is bad. 

(ii) Suppose S is not a minimal stopping set and suppose S does not contain any problematic nodes. Then, 
any node in S belongs to a smaller stopping set within S. We claim that each node within S belongs 
to a minimal stopping set within S. Otherwise, a node belongs to a proper non-minimal stopping set 
S' C S and is not contained in any smaller stopping set within S' - thereby, implying that the node is 
a problematic node. Therefore, all nodes in S are contained in minimal stopping sets within S. 
To prove the last part of the theorem, suppose one of these minimal stopping sets, say Sj, is not the 
support of any non-zero codeword in C. Then, there exists a bad nc-pseudocodeword p = (pi, . . . ,p n ) 
with support S, where pi = x, for an appropriately chosen positive even integer x that is at least 4, for 
% G Sj and pi = 2 for i G S — Sj, and pi = for i ^ S. Let i* be the index of a variable node v* in 
Sj. If there are distinct codewords c^, c^ 2 \ . . . , whose supports contain v* and whose supports 
are contained in S, then let %\, %2, if be the indices of variable nodes in the supports of these 



W: 
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codewords outside of Sj. (Note that we choose the smallest number t' < t of indices such that each 
codeword contains one of the variable nodes v^, . . . ,Vi t , in its support.) The following weight vector 
w = (wi, . . . , w n ) ensures that p is a bad pseudocodeword. 

-1 if i = i* 

+1 if i = i 1 ,i 2 ,---,H' 
M if * S 

otherwise 

(Note that x can be chosen so that pw T < and it is clear that cw T > for all codewords in the code.) 
Now suppose Sj contains the support of a codeword c' and has property ©, then we can construct a bad 
pseudocodeword on Sj using the previous argument and that in part 2(a)(i) above (since Sj is minimal) 
and allow the remaining components of p in S — Sj to have a component value of 2. It is easy to verify 
that such a pseudocodeword is bad. 

Conversely, suppose every minimal stopping set Sj within S does not have property and contains 
the support of some non-zero codeword c^') within it. Then, this means that Sj = supp(c^)) and that 
between every pair of nodes within Sj there is a path that contains only degree two check nodes in G\ s . 
Then, for any pseudocodeword p with support S, there is a decomposition of p, as in Lemma 15.21 such 
that p can be expressed as a linear combination of codewords c^' 's. Thus, by Theorem 15.11 there are 
no bad pseudocodewords with support S. 



Proof of Lemma [Or Suppose S is minimal and does not have property B. Then each pair of nodes in S 
are connected by a path via degree two check nodes, and hence all components of a pseudocodeword p on S 
are equal. If S is the support of a codeword c, then any pseudocodeword is a multiple of c, and so ts = 1. 
If S is not the support of a codeword, then the all-two's vector is the only irreducible pseudocodeword, and 
ts = 2. ■ 



Proof of Theorem I5.4| - Let the LDPC code C represented by the LDPC constraint graph G have blocklength 
n. Note that the weight vector w = (wi,w<z, . . . ,w n ) has only +1 or —1 components on the BSC, whereas 
it has every component Wi in the interval [—L,+L] on the truncated AWGN channel TAWGN(L), and has 
every component Wi in the interval (— oo,+oo) on the AWGN channel. That is, 

p Esc{ G ) = {Pl 3w G {+1,-1}" s -t- P wT < 0,cw T > 0,V0 + c G C}. 

P TAWGN(L)( G ) = {Pl 3w e [+L,-L] n s.t. pw T < 0,cw T > 0,V0 + c G C}. 

p AWGn(G) = {Pl^w G (-oo,+oo) n s.t. pw T < 0,cw T > 0,V0 + c G C}. 
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It is clear from the above that, for L > 1, PEsc( G ) ^ P tawgn(l)( G ) ^ P awgn( G )- 
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